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ABSTRACT 

This document is an assessment of the effectiveness 
of the Policy/School Liaison Program (PSLP), which began in 1982. 
PSLP offered 11th and 12th graders a one-semester, law-related 
education course. The 1985-86 PSLP was designed to expand the program 
and to develop materials necessary to expedite program 
institutionalization. A curriculum manual and a videotape for 
classroom use was developed in July, 1986. The report consists of 
four chapters. Chapter one contains program background and scope of 
the evaluation. Chapter two is a description of program 
implementation. Chapter three is an analysis of student outcomes for 
the attitude survey and criminal justice examination. Chapter four 
contains conclusions and recommendations. The findings indicate that 
the program met its objectives of establishing partnerships with 
educational agencies; training teachers and police officers in course 
content, methods, and materials; and developing curriculum materials, 
but did not meet its objective of improving student attitudes toward 
the legal system. Recommendations include: (1) prioritizing course 
requirements; and (2) using a control group of non-program students 
to assess change in student attitudes. Appended is an item analysis 
of the PSLP Attitude Survey and Criminal Justice Examination. (SM) 
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SUMMARY 



The 1985-86 Police/School Liaison Program (P.S.L.P.) offered 
eleventh and twelfth graders a one-semester law-related education 
course. A social studies teacher and a police officer from the 
local precinct team taught one or two classes at each of 63 
public and eight private high schools. Approximately 3,000 
students took part each term. In January and July, 1986, teachers 
and police officers participated in staff training workshops. 

Conclusions about the success of P.S.L.P. in meeting its 
objectives are limited by the fact that the Office of Educational 
Assessment/High School Evaluation Unit could only assess the 
program retrospectively. The program met its objectives of estab- 
lishing partnerships with educational agencies, training teachers 
and police officers in course content, methods, and materials, 
and developing curriculum materials and did not meet its objec- 
tive of improving students 1 attitudes toward the legal system. 

On a program-developed and administered attitude survey and 
content test, students did not show any statistically-significant 
improvement in their attitudes toward the legal system. Although 
program students improved their knowledge of legal issues, 
testing procedures made the meaning of these findings ambiguous. 
Flaws in the attitude survey itself and in the administration of 
both the survey and the criminal justice examination raised 
questions about the validity of survey and examination results. 
The failure to have a control group of non-program students 
complete the survey and the knowledge examination made it 
impossible to assess whether changes were due to program 
participation or to other unrelated factors. 

Although the program's nearly 1,000-page guide contains a 
wealth of useful material, it has shortcomings. While allowing 
teachers opportunities for flexibility and creativity, the guide 
does not give a clear shape to the course. It does not indicate 
which of its many lessons are priorities and which should be only 
supplemental, it contains lessons with so much content that they 
are actually units which could take from several days to several 
weeks to teach. The videotape lacked certain production qualities 
needed to convey its incidents in a way that was both clear and 
involving for students. The.je problems reduce its effectiveness 
as a stimulus for classroom discussion. 

On the basis of the evaluation findings, the following 
recommendations are aimed at enhancing program benefits: 

o The curriculum guide should be revised to stipulate course 
priorities and requirements and indicate the approximate 
number of class periods needed to complete each lesson. 

o Any attempts to assess changes in students' attitudes or 
knowledge should involve both program students and a 
control group of non-program students. 
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I. INTRODUCTION 

PROGRAM BACKGROUND 

The Police/School Liaison Program (P.S.L.P.), which began in 
1982, offered eleventh and twelfth graders a one-semester, law- 
related education course. A social studies teacher and a police 
officer from the local precinct teamed to teach one or two 
classes at each of 63 public and eight private high schools. 
Approximately 3,000 students took part each term. In January and 
July, 1986, teachers and police officers participated in staff 
training workshops. After the end of the program year, staff 
completed a curriculum manual and a videotape for classroom use. 

During the summer of 1982, P.S.L.P. developed an 800-page 
curriculum for use in its social studies elective classes. In 
1982 through 1985, the P.S.L.P. operated in 34 New York City 
public and private high schools. The 1985-86 P.S.L.P. was 
designed to expand the program and to develop material necessary 
to expedite program institutionalization. 

PROGRAM OBJECTIVES 

The original goals of the program, which were based on a 

proposed start-up date of August, 1985, were as follows: 

o to establish partnerships with public and private 

educational agencies promulgating law-related education; 

o to develop student knowledge about the U.S. system of law 
and law enforcement agencies; 

o to improve student attitudes toward school, the law, and 
law enforcement agencies; 

o to train teachers and police officers in the P.S.L.P. ! s 
course content, methods, and materials; 

, ERIC 7 



O.E.A./H.S.E.U. staff analyze the statistical reliability of 
these measures. 

SCOPE OF THIS REPORT 

The O.E.A./H.S.E.U. report of the 1985-86 P./S.L.P. 
evaluation consists of four chapters. Chapter I contains program 
background and scope of the evaluation. Chapter II is a 
description of program implementation. Chapter III is an analysis 
of student outcomes for the attitude survey and criminal justice 
examination. Chapter IV contains conclusions and recommendations. 
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II. PROGRAM IMPLEMENTATION 

PROGRAM OVERVIEW 

The P.S.L.P. proposal stipulated that the program begin in 
summer, 1985 with the development of curriculum which would then 
be field tested and distributed during the 1985-86 school year. 
The late arrival of funding altered this sequence. The program 
actually operated from October 1, 1985 to September 30, 1986. 
Classes took place prior to the development of the new 
curriculum. Teachers, therefore, had to rely on previously- 
developed curriculum, rather than being able to field test new 
material. Curriculum development was postponed until summer, 
1986. 

According to program records, P.S.L.P. classes met in 63 
public high schools. At 51 percent of these schools, a team 
consisting of a police officer and a social studies teacher 
offered two P.S.L.P. classes; at the rest of the schools, they 
offered a single class. Assigned full-time to a school, each 
police officer co-taught a P.S.L.P. class three or four times a 
week. The remainder of their time was spent providing a police 
presence in the school. Approximately 3,000 students participated 
each term. The number of students registered for each class 
ranged from 15 to 68, with an average class register of 32. 

One program goal was to establish partnerships with 
educational agencies and public and private agencies promulgating 
law-related education. O.E.A./H.S.E.U. could only assess the 
program's success in meeting this goal retrospectively on the 
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basis of information provided by the program director. He 
reported that P./S.L.P. established partnership activities with 
the New York City Police Department, the New York City 
Partnership, National Institute for Citizen Education and the 
Law, the New York City Coalition for Law and Citizenship 
Education, John Jay College of Criminal Justice, and Queens 
College Law School. He indicated that these agencies and 
institutions played various roles in relation to the project. The 
New York City Police Department, for example, contributed the 
services of the program director and the police officers. Queens 
College Law School interns worked at approximately ten high 
schools, where they did legal research and set up mock trials to 
assist the teacher and police officer in the classroom. 

TRAINING 

As a result of the delay in funding, it was not possible for 
the program staff to conduct the training that had been scheduled 
for the summer prior to the start cf the 1985-86 school year. As 
a result, classes began a full semester prior to the first 
training session. In January, 1986, new officers met for five 
days of instructional training and all participating teachers and 
police officers met for three training sessions at the same site. 
During July, 1986, teachers and police officers met for four days 
at John F. Kennedy High School. Thus, training originally 
intended to precede the program year became, out of necessity, 
training for the following year. 

5 
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An O.E. A./H.S.E.U. evaluator attended one day of this four- 
day curriculum development/staff training workshop. About 50 
police officers and teachers met for a combination of 
orientation, general staff development, and training in the use 
of curriculum materials, which were under development at that 
time. The evaluator observed a series of large-group 
presentations on specific lesson plans and larger curriculum 
focal points. Some of the speakers were police administrators 
dealing with non-curriculum issues. A small group was working at 
developing lesson plans and revising a cognitive test developed 
to assess knowledge of course content:. Much of the curriculum 
development had been done earlier, a lot of it during the actual 
conduct of the course. 

There was considerable discussion. A steady theme revolved 
around the belief expressed by some of the police officers that 
the primary objective of the course was to get the students to 
adopt a point of view sympathetic to the police. They defined 
their concerns as lack of cooperation from judges, inefficiency 
of the criminal justice system, criticisms of the police by the 
media and the public, and hostility of racial minorities toward 
the police. Several expressed puzzlement at how little change 
seemed to have taken place in the students' attitudes as a result 
of the courses. A few of the teachers maintained that they were 
trying to get the students to think about police-related issues 
with more information, sophistication, and objectivity. Most of 
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the participants who spoke were extremely positive about the 
experience . 

CURRICULUM DEVELOPMENT 

During the summer of 1986, curriculum and video production 
commi tees met four days a week for several weeks. These 
committees consisted of 17 police officers, nine teachers, and 
two assistant principals for social studies. Video committee 
members developed and produced a videotape. Curriculum committee 
members revised previously-developed curriculum material and 
developed new written material. They produced the nfcarly-1,000 
page American Government and Criminal Justice System Curriculum 
Guide and an hour-long videotape. 

The videotape contains seven short segments. They involve 
such issues as due process, different kinds of criminal offenses, 
and police procedures. Each segment concludes with a series of 
questions for classroom discussion. 

It is clear that a great deal of work went into the video 
production effort. In general, the incidents depicted are 
relevant to high school students, involving characters from their 
age group, and often utilizing a school or school grounds 
setting. However, the final product does not reflect professional 
video expertise. Several segments are visually static and are 
edited in shots that are too long for optimal retention of viewer 
interest. The visual narrative is at times ambiguous even when 
coupled with an explanatory sound track. 
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The curriculum guide contains 85 lessons. It includes charts 
and graphs, newspaper and magazine articles, selected readings 
from books, and extensive excerpts from the Police Student f s 
Guide ; most of these are followed by discussion questions. Each 
lesson contains at least two performance objectives, several 
central concepts, a content outline, and a listing of suggested 
activities. 

The curriculum guide is divided into seven units. The first 
four deal with comparative government, the U.S. Constitution, and 
state and local government. Sixty percent of the guide consists 
of two units: Criminology and Criminal Law, and Role of the 
Police in Our Criminal Justice System. A final unit describes the 
School Program to Educate and Control Drug Abuse. The appendices 
include suggested themes for law and criminal justice papers, a 
glossary, a bibliography, and recommended audio-visual materials. 

The guide contains a great deal of potentially useful 
material. The introduction, for example, discusses the inquiry or 
problem-solving approach to teaching which encourages students to 
be actively involved in the learning process. The units which 
follow include reprints and other curriculum material designed to 
stimulate student thinking about legal issues. 

The guide's format, however, does not facilitate its use as 
a curriculum manual or resource. Its table of contents lists the 
titles of the seven units, but not the contents of each. As a 
result, teachers have no ready way to locate specific lessons or 
reprints within units whose length ranges from 30 to 250 pages. 
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The guide is more a compendium of materials than a 
curriculum manual. It contains more lessons than there are class 
sessions in a semester and each lesson contains more material 
than can be covered in a single class session. This wealth of 
material could enable teachers to probe deeply into any number of 
issues. However , it does not provide guidance as to which lessons 
are most important to cover or what approximate percentage of 
lessons from each unit should be taught. Teachers are given so 
many options that they could construct many different curricula 
for their students that would not necessarily cover a common set 
of course objectives. 
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III. STUDENT OUTCOMES 

ATTITUDE SURVEY 

P.S.L.P. staff developed a student survey consisting of 25 
positive statements regarding criminal justice agencies* It 
contained a five-point scale ranging from "strongly agree" (one) 
to "strongly disagree" (five). Program staff administered the 
same survey prior to classroom instruction (pretest) and at the 
end of the course (posttest). Student names appeared on the 
completed surveys. The P.S.L.P. administrator supplied 
O.E.A./H.S.E.U. with data for 221 program students from seven 
schools. Forty-three percent of these students had both pretest 
and posttest survey data. Only these students could be included 
in analyses of attitude changes. 

In general, students expressed favorable attitudes toward 
criminal justice agencies. The average rating was 2.3 on the 
pretest and 2.2 on the posttest.* The mean difference between the 
attitude survey pretest and posttest was not statistically 
significant (t = 1.38; df = 95; p > .05). These responses 
indicate that, on average, students agreed with the positive 
statements in the attitude survey on both pre- and posttests. 
Students' most common rating (35 percent) on both pretest and 
posttest was "agree" (two). A response of "disagree" (four) was 

*O.E.A./H.S.E.U. computed reliability coefficients for the 
pretest and posttest separately for the students with complete 
survey data. Using a measure of internal consistency, alpha, the 
reliability coefficients were determined to be high, (r ■ .79) 
for the pretest and (r = .80) for the posttest, indicating that, 
in general, the 25 survey items were measuring the same thing. 
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uncommon (nine percent) and, with one exception, a response of 
"strongly disagree" (five) was very rare (three percent).* 

Most of the items did not show a positive response shift. 
However, the lack of statistical significance indicates that 
these shifts might have been due to chance variation. Certainly, 
students' attitude might have changed as a result of the course. 
Other explanations are possible, however. Because pretest 
responses were so positive, little room was left for positive 
change on the posttest. 

CRIMINAL JUSTICE EXAMINATION 

P.S.L.P. staff also developed a single criminal justice 
examination which staff used as pretest and posttest. Staff 
administered the test to program students only; they did not give 
it to a comparison group of students who did not take the 
P.S.L.P. course. Of the program students in the sample (N = 221), 
48 percent had both pretest and posttest scores, student 
performance on the 50-item examination improved significantly 
from an average pretest score of 21.6 to an average posttest 
score of 32.8 (t = 13.24; df = 106; p_ > .05). Although this 
improvement is impressive, without results from a comparison 
group of non-program students, it is impossible to determine 
whether the improvement is the result of classroom experience. 

**Students responded "strongly disagree" 28 percent of the time 
to the statement, "Some day I would like to be a police officer." 
More students selected this response on the pretest (31 percent) 
than on the posttest (26 percent). This item represents the only 
shift of five percent in the direction of "strongly disagree." 
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IV. CONCLUSIONS AND RECOMMENDATIONS 

Conclusions about the success of P./S.L.P. in meeting its 
objectives are limited by the fact that O.E.A./H.S.E.U. could 
only assess the program retrospectively. The program met its 
objectives of establishing partnerships with educational 
agencies, training teachers and police officers in course 
content, methods, and materials, and developing curriculum 
materials. 

Although the program 9 s nearly 1,000-page guide contains a 
wealth of useful material, it has shortcomings. While allowing 
teachers opportunities for flexibility and creativity, the guide 
does not give a clear shape to the course. The result is a 
compendium of material that leaves teachers on their own to make 
such potentially disparate decisions that courses taught at 
different schools might bear little resemblance to each other. 

The videotape lacked certain production qualities needed to 
convey its incidents in a way that was both clear and involving 
for students. These problems reduce its effectiveness as a 
stimulus for classroom discussion. 

Students did not show any statistically-significant 
improvement in their attitudes toward the criminal justice 
system. Although program students improved their knowledge of 
legal issues, testing procedures did not allow for clear causal 
attribution of this improvement to course participation. Flaws in 
the attitude survey itself and in the administration of both the 
survey and the criminal justice examination raised questions 
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about the validity of survey and examination results. The 
inclusion in the survey of only positive statements about law 
enforcement agencies may have suggested to students that there 
was a "correct" or "acceptable" response to each. The lack of 
anonymity on attitude surveys which were collected by the 
authority figures who graded students further compromised any 
effort to get honest responses. The failure to have a control 
group of non-program students complete the survey and the 
knowledge examination made it impossible to assess whether 
changes were due to program participation or to other unrelated 
factors. 

On the basis of the evaluation findings, the following 

recommendations are aimed at enhancing program benefits and 

facilitating their assessment: 

o The curriculum guide should be revised to stipulate course 
priorities and requirements and indicate the approximate 
number of class periods needed to complete each lesson. 

o The curriculum guide should be bound in separate units, 
each preceded by a complete table of contents. 

o Any attempts to assess changes in students 1 attitudes or 
knowledge should involve both program students and a 
control group of non-program students. 

o Students should retain their anonymity when filling out 
attitude surveys. 

o Attitude surveys should contain statements keyed in both a 
positive and a negative direction. 
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APPENDIX 

ITEM ANALYSIS OF P./S.L.P. ATTITUDE SURVEY 
AND CRIMINAL JUSTICE EXAMINATION 

As part of its 1985-86 P./S.L.P. evaluation, O.E.A./H.S.E.U. 

examined the reliability of the program's attitude survey and 

examination. These analyses are summarized in this appendix. 

ATTITUDE SURVEY 

P./S.L.P. staff developed an attitude survey consisting of 
25 positive statements regarding criminal justice agencies and 
using a five-point scale ranging from "strongly agree" (one) to 
"strongly disagree" (five). They administered the same survey as 
pretest and posttest. Ninety-six students had both pretest and 
posttest survey data and could therefore be included in the 
O.E.A./H.S.E.U. analysis. 

Because the posttest is designed to represent changes in 
student attitudes which result from the program, the reliability 
of individual items was examined in detail for the posttest. 
O.E.A./H.S.E.U. performed a statistical analysis whereby each of 
the 25 survey items was correlated with the remaining 24. For 
example, item one was correlated with items two through 25 
combined. The results of this analysis, designated the corrected 
item-total correlation, are presented in Appendix Table 1 for 
each of the 25 survey items. O.E.A./H.S.E.U. determined that any 
item with a corrected item-total correlation below .30 was too 
weak for internal consistency purposes. As indicated in Appendix 
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APPENDIX TABLE 1 

Attitude Survey Posttest: Corrected 
Item-Total Correlation for 25 Survey Items 



Survey Item a 


Correlation 


Coefficient 


1 


.50 


2 


.47 


3 


.46 


4 


.62 


5 


.33 


6 


.36 


7 


.51 


8 


.39 


9 


.64 


10 


.43 


11 


.45 


12 


.33 


13 


.38 


1 A 


.35 


15 


.35 


16 


.26* 


17 


.13* 


18 


.26* 


19 


.37 


20 


.19* 


21 


.43 


22 


.32 


23 


.23* 


24 


.15* 


25 


.13* 



a Students rated each item on a five-point 
scale ranging from "strongly agree" (one) to 
"strongly disagree" (five). 



*An asterisk indicates a correlation below the 
suggested .30 cut-off point for inclusion in 
the survey. 

• Seven items had corrected-item 
total correlations below .30, the 
suggested cut-off for inclusion in 
the survey. 
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Table 1, seven items (16, 17, 18, 20, 23, 24, and 25) had 
corrected item-total correlations below .30. 

The mean responses for the attitude survey pretest are 
presented in Appendix Table 2. A number of items (5, 11, 13, 15, 
17, 18, 23, 24) with mean ratings between "strongly agree" (one) 
and "agree" (two) were represented by a mean rating of less than 
2.0. 

Some items provided poor measures of student attitudes. Item 
15, for example ("The police try to decrease the criminal 
behavior in the streets") is a statement of the obvious. Although 
some people may believe that police make no effort, it would be 
surprising if most people did not agree with this statement. Item 
11 ("If I saw a crime committed, I would report it to the 
police") is an example of an item which suggests a positive 
response. Regardless of whether these students would call the 
police to report a crime, it looks good to say that they would. 

CRIMINAL JUSTICE EXAMINATION 

O.E.A./H.S.E.U. conducted an item analysis of the criminal 
justice examination for the 107 students with complete pretest 
and posttest data. All but three of the 50 test items were four- 
choice multiple choice questions; the others were five-choice 
questions, it was expected that students would answer 47 items 
correctly approximately 25 percent of the time by chance alone. 
Students demonstrated some knowledge of the material; prior to 
any classroom experience, with better than chance performance 
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APPENDIX TABLE 2 

Mean Responses 
for the Attitude Survey Pretest 



Survey Item 


Mean a 


1 


2.5 


2 


2.3 


3 


2.9 


4 


2.6 


5 


1.9* 


6 


2.8 


7 


2.7 


8 


2.4 


9 


2.1 


10 


2.5 


11 


1.9* 


12 


2.8 


13 


1.8* 


1 A 


3.3 


15 


1.9* 


16 


2.5 


17 


1.8* 


18 


1.7* 


19 


2.4 


20 


2.0 


21 


2.2 


22 


2.0 


23 


1.9* 


24 


1.9* 


25 


2.4* 



a Students rated each item on a five-point 
scale ranging from "strongly agree" (one) to 
"strongly disagree" (five). 



* An asterisk indicates a mean pretest rating 
below the suggested 2.0 cut-off point for 
inclusion in the survey. 

• Eight items had mean pretest 
ratings less than 2.0, the 
suggested cut-off point for 
inclusion in the survey. 
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on the pretest; on the average, students answered 43 percent of 
the questions correctly. 

Examination performance for the 50 items on the pretest is 
illustrated in Appendix Table 3. Students correctly answered 
several items most of the time on the pretest, indicating that 
perhaps these were easy questions. Students correctly answered 22 
of the 50 items more than half of the time. They correctly 
answered one item 88 percent of the time on the pretest; they 
knew that a person may not use physical force to resist arrest 
under any circumstance. O.E.A. /H.S.E.U. identified items (1, 13, 
14 , 15, 41, 44, 45, 48) which students answered correctly over 67 
percent of the time as "too easy." 

Examination of performance for the 50 posttest items are 
provided in Appendix Table 4. Near or below chance performance 
was evident for only one posttest item. Students answered item 
nine correctly only 22 percent of the time on the posttest; 
students had apparently not learned that "Bill could not be 
charged with a crime for recklessly damaging a $200 window." The 
students' lack of knowledge of this material might be due not to 
a bad test item, but to the way in which teachers presented this 
information. 

The change in the percentage of students answering the 
examination questions correctly from pretest to posttest is 
represented in Appendix Table 5. For two of the items, perfor- 
mance was considerably worse on the posttest than on the pretest. 
Students correctly answered item 15, which pertained to larceny, 
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APPENDIX TABLE 3 







Summary of 


Students ' 


Responses 


to Each 


of the 










Pretest 


Exam Items 








Item 


Number 


Number 


Percent 




Number 


Number 


Percent 


Correct 


Incorrect 


Correct 


Item 


Correct 


Incorrect 


Correct 


1 


78 


29 


72.9* 


26 


32 


75 


29.9 


2 


66 


41 


61.7 


27 


56 


51 


52.3 


3 


48 


59 


44.9 


28 


46 


61 


43.0 


4 


19 


88 


17.8 


29 


23 


84 


21.5 


5 


48 


59 


44.9 


30 


29 


78 


27.1 


6 


55 


52 


51.4 


31 


45 


62 


42.1 


mm 

7 


46 


61 


43.0 


32 


47 


60 


43.9 


8 


67 


40 


62.6 


33 


44 


63 


41.1 


9 


3 


104 


2.8 


34 


33 


74 


30.8 


10 


48 


59 


44.9 


35 


65 


42 


60.7 


11 


40 


67 


37.4 


36 


46 


61 


43.0 


12 


67 


40 


62.6 


37 


71 


36 


66.4 


13 


94 


13 


87.9* 


38 


69 


38 


64.5 


14 


73 


34 


68.2* 


39 


11 


96 


10.3 


15 


25 


82 


76.6* 


40 


18 


89 


16.8 


16 


23 


84 


21.5 


41 


76 


31 


71.0* 


17 


41 


66 


38.3 


42 


70 


37 


65.4 


18 


50 


57 


46.7 


43 


61 


46 


57.0 


19 


21 


86 


19.6 


44 


74 


33 


69.2* 


20 


21 


86 


19.6 


45 


76 


31 


71.0* 


21 


59 


48 


55.1 


46 


26 


81 


24.3 


22 


48 


59 


44.9 


47 


62 


45 


57.9 


23 


10 


97 


9.3 


48 


79 


28 


73.8 


24 


44 


63 


41.1 


49 


17 


90 


15.9 


25 


29 


78 


27.1 


50 


9 


98 


8.4 



*An asterisk indicates that at least 67 percent of the students answered the 
item correctly. 

• Over two-thirds of the students answered six items on the pretest 
correctly, suggesting that these items may be "too easy." 
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APPENDIX TABLE 4 



Summary of Students' Responses to Each of the 
Posttest Exam Items 



TP ^ 

Item 


Number 


Number 


Percent 




Number 


Number 


Percent 


Correct 


Incorrect 


Correct 


Item 


Correct 


Incorrect 


Correct 


1 


ft i 

91 


16 


85.0 


26 


75 


32 


70.1 


2 


ft A 

80 


27 


74.8 


27 


72 


35 


67.3 


3 


74 


33 


69.2 


28 


65 


42 


60.7 


4 


63 


44 


58.9 


29 


42 


65 


39.3 


c 

b 


ft r\ 

80 


27 


74.8 


30 


51 


56 


47.7 


6 


ft o 
88 


19 


82.2 


31 


61 


46 


57.0 


7 


78 


29 


72.9 


32 


67 


40 


62.6 


8 


ft ^ 
93 


14 


86.9 


33 


64 


43 


59.8 


9 


ft A 

24 


83 


22.4* 


34 


61 


46 


57.0 


10 


78 


29 


72.9 


35 


87 


20 


81.3 


11 


74 


33 


69.2 


36 


70 


37 


65.4 


1 ft 

12 


n ft 
92 


15 


86.0 


37 


88 


19 


82.2 


13 


95 


12 


88.8 


38 


81 


26 


75.7 


14 






00 • u 


iy 




54 


49.5 


15 


55 


52 


51.4 


40 


43 


64 


40.2 


16 


71 


36 


66.4 


41 


83 


24 


77.6 


17 


59 


48 


55.1 


42 


89 


18 


83.2 


18 


72 


35 


67.3 


43 


83 


24 


77.6 


19 


42 


65 


39.3 


44 


92 


15 


86.0 


20 


60 


47 


56.1 


45 


82 


25 


76.6 


21 


90 


17 


84.1 


46 


53 


54 


49.5 


22 


63 


44 


58.9 


47 


76 


31 


71.0 


23 


62 


45 


57.9 


48 


64 


43 


59.8 


24 


72 


35 


67.3 


49 


52 


55 


48.6 


25 


65 


42 


60.7 


50 


40 


67 


62.6 



*An asterisk indicates that fewer than one-fourth of the students 
answered this item correctly. 

• Students answered item nine correctly less than one-fourth 
of the time on the posttest, suggesting that it might be 
removed from the examination. 
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APPENDIX TABLE 5 

Net Change from Examination Pretest to Posttest 
of Percentage of Students Answering 
Items Correctly 





Item 


% Change 


Item 


% Change 



1 


12.1 


26 


40.2 


2 


13.1 


27 


15.0 


3 


24.3 


28 


17.7 


4 


41.1 


29 


17.8 


5 


29.9 


30 


20.6 


6 


30.8 


31 


14.9 


7 


29.9 


32 


18.7 


8 


24.3 


33 


18.7 


9 


19.6 


34 


26.2 


10 


28.0 


35 


20.6 


11 


31.8 


36 


22.4 


12 


23.4 


37 


15.8 


13 


0.9 


38 


11.2 


14 


17.8 


39 


39.2 


15 


-25.2 


40 


23.4 


16 


44.9 


41 


6.6 


17 


16.8 


42 


17.8 


18 


20.6 


43 


20.6 


19 


19.7 


44 


16.8 


20 


36.5 


45 


5.6 


21 


29.0 


46 


25.2 


22 


14.0 


47 


13.1 


23 


48.6 


48 


-14.0 


"4 


26.2 


49 


32.7 


25 


33.6 


50 


54.2 



• Performance on items 15 and 48 was 
considerably worse on the posttest 
than on the pretest. 
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over 76 percent of the time on the pretest and 51 percent of the 
time on the posttest. Students correctly answered item 48, which 
pertained to sexual misconduct, 74 percent of the time on the 
pretest and 60 percent of the time on the posttest. Based on 
pretest data, O.E.A./H.S.E.U. classified these two item as "too 
easy. 11 It is not clear why students had more difficulty with 
these items on the posttest; perhaps a considerable number made 
lucky guesses on the pretest. 

SUGGESTED CHANGES 
Attitude Survey 

Survey items with pretest mean ratings near 2.5, the 
midpoint of the scale should be maintained because they allow 
room for change on the posttest which lends itself to an 
evaluation of the influence of the course. The survey items that 
correlate best with the remaining items on the posttest should be 
maintained since they are the items responsible for high internal 
consistency. The items with a corrected item-total below .30 and 
those with a mean pretest rating less than 2.0 should be 
eliminated from the survey. 

Half of the items should be keyed in the negative direction. 
Instead of "strongly agree 11 representing the most favorable 
response for all of the items, half of them should have "strongly 
disagree" as the most favorable response to the criminal justice 
agencies. For example, the statement that "All police are lazy" 
would require a negative response if students wanted to respond 
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favorably toward police. This would control for possible response 
bias in that some students tend to be agreeable. 

The midpoint response, "somewhat agree , M should be 
eliminated. It is not a true midpoint since it is a statement of 
agreement. In addition, an even number of possible choices would 
eliminate the possibility of "fence sitting"; there are always 
people who do not wish to commit themselves one way or another. 

Finding 25 items that meet all of the above objectives can 
be difficult. One method of finding the best 25 items would 
include pilot testing. Perhaps about 100 questions could be given 
to a large number of students in sets of 25. These students need 
not be part of the program, but they should be as similar as 
possible to the students who do end up in the program (e.g., same 
grade). These items can be evaluated for reliability, and to see 
if they have an average value near the midpoint of the scale. The 
best 25 can be retained and P./S.L.P. can be evaluated 
appropriately. 

Criminal Justice Examination 

The items which students answered correctly on the pretest 
over 67 percent of the time should be removed from the 
examination. O.E.A./ H.S.E.U. analysis indicates that these 
questions are too easy. 

There appears to be no advantage to pretesting students on 
material they have not been taught. The best way to see if 
program students have learned the material as a result of program 
participation is to test them and a control group of non-program 
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ERLC 



£9 



students once at the end of the program. Participating students 
would be expected to outperform non-program students. However, it 
should be made certain that the control subject are equated with 
the course students for grade and academic ability; otherwise an 
unfair comparison might result. 
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